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PACKETIZING DEVICES FOR 
SECURE SCALABLE DATA STREAMING 

CROSS REFERENCE TO RELATED APPLICATION 
5 This Application is a Continuation-in-Part of the co-pending, commonly- 

owned US Patent Application, Attorney Docket No. HP1 001 4738, Serial No. 

, filed May 4, 2001 , by S. J. Wee et al., and entitled "Encoding 

and Decoding Methods for Secure Scalable Streaming and Related 
Systems." 

10 

TECHNICAL FIELD 

The present claimed invention relates to the field of streaming media 
data, particularly scalably encoded and progressively encrypted data. More 
specifically, the present claimed invention relates to the packetizing of such 
15 data. 

BACKGROUND ART 

Wireless streaming environments present many challenges for the 
system designer. For instance, clients can have different display, power, 

20 communication, and computational capabilities. In addition, wireless 

communication links can have different maximum bandwidths, quality levels, 
and time-varying characteristics. A successful wireless video streaming 
system must be able to stream video to heterogeneous clients over time- 
varying wireless communication links, and this streaming must be performed 

25 in a scalable and secure manner. Scalability is needed to enable streaming 
to a multitude of clients with different device capabilities. Security is 
particularly important in wireless networks to protect content from 
eavesdroppers. 

30 In order to achieve scalability and efficiency in wireless streaming 

environments, one must be able to easily adapt or transcode the compressed 
video stream at intermediate network nodes. A transcoder takes a 
compressed video system as the input, then processes it to produce another 
compressed video stream as the output. Sample transcoding operations 

35 include bitrate reduction, rate shaping, spatial downsampling, frame rate 
reduction, and changing compression formats. Network transcoding can 
improve system scalability and efficiency, for example, by adapting the spatial 
resolution of a video stream for a particular client's display capabilities or by 
dynamically adjusting the bitrate of a video stream to match a wireless 

40 channel's time-varying characteristics. 
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While network transcoding facilitates scalability in video streaming 
systems, it also presents a number of challenges. First, while computationally 
efficient transcoding algorithms have been developed, even these are not 
5 well-suited for processing hundreds or thousands of streams at intermediate 
wired network nodes or even a few streams at intermediate low-power 
wireless networking relay nodes. Furthermore, network transcoding poses a 
serious threat to the security of the streaming system because conventional 
transcoding operations performed on encrypted streams generally require 
10 decrypting the stream, transcoding the decrypted stream, and then re- 
encrypting the result. Because every transcoder must decrypt the stream, 
each network transcoding node presents a possible breach in the security of 
the entire system. 

15 More specifically, in conventional video streaming approaches 

employing application-level encryption, video is first encoded into a bitstream 
using interframe compression algorithms. These algorithms include, for 
example, the Moving Picture Experts Group (MPEG) standard, the 
International Telecommunications Union (ITU) standard, H.263, or intraframe 

20 compression algorithms such as, for example, the Joint Photographic Experts 
Group (JPEG) or JPEG2000 standards. The resulting bitstream is then 
encrypted, and the resulting encrypted stream is packetized and transmitted 
over the network using a transport protocol such as user datagram protocol 
(UDP). Prior Art Figure 1 is a block diagram 100 which illustrates the order in 

25 which conventional application-level encryption is performed (e.g., Encode 
102, Encrypt 104, and Packetize 106). One difficulty with this conventional 
approach arises when a packet is lost. Specifically, error recovery is difficult 
because without the data from the lost packet, decryption and/or decoding 
may be difficult if not impossible. 

30 

Prior Art Figure 2 is a block diagram 200 illustrating another 
conventional secure video streaming system that uses network-level 
encryption (e.g., Encode 202, Packetize 204, and Encrypt 206 ). The system 
of Prior Art Figure 2 can use the same video compression algorithms as the 
35 system of Prior Art Figure 1 . However, in the system of Prior Art Figure 2, the 
packetization can be performed in a manner that considers the content of the 
coded video and thus results in better error recovery, a concept known to the 
networking community as application-level framing. For example, a common 
approach is to use MPEG compression with the real-time transport protocol 
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(RTP) which is built on UDP. RTP provides streaming parameters such as 
time stamps, and suggests methods for packetizing MPEG payload data to 
ease error recovery in the case of lost or delayed packets. However, error 
recovery is still difficult and, without data from a lost packet, decryption and/or 
5 decoding is still difficult if not impossible. 

Both of the conventional approaches of Prior Art Figure 1 and Prior Art 
Figure 2 are secure in that they transport the video data in encrypted form. 
However, with these conventional approaches, if network transcoding is 
needed, it must be performed in accordance with the method of Prior Art 
Figure 3. That is, as shown in block diagram 300, the necessary transcoding 
operation is a decrypt 302, decode 304, process 306, re-encode 308, and re- 
encrypt 310 process. As shown in the block diagram 400 of Prior Art Figure 4, 
in another conventional approach, the computational requirements of the 
operation of Prior Art Figure 3 are reduced to a decrypt 402, transcode 404, 
and re-encrypt 406 process. Specifically, this computational reduction is 
achieved by incorporating an efficient transcoding algorithm (e.g., transcode 
module 404) in place of the decode 304, process 306, and re-encode 308 
modules of Prior Art Figure 3. However, even such improved conventional 
transcoding algorithms have computational requirements that are not well- 
suited for transcoding many streams in a network node. Furthermore, a more 
critical drawback stems from the basic need to decrypt the stream for every 
transcoding operation. As mentioned above, each time the stream is 
decrypted, it opens another possible attack point and thus increases the 
vulnerability of the system. Thus, each transcoder further threatens the 
security of the overall system. 

As yet another concern, wireless streaming systems are limited by 
wireless bandwidth and client resources. Wireless bandwidth is scarce 
30 because of its shared nature and the fundamental limitations of the wireless 
spectrum. Client resources are often practically limited by power constraints 
and by display, communication, and computational capabilities. As an 
example, wireless transmission and even wireless reception alone typically 
consume large power budgets. In order to make the most efficient use of 
35 wireless bandwidth and client resources, it is desirable to send clients the 
lowest bandwidth video streams that match their display and communication 
capabilities. In wireless streaming systems where a sender streams video to a 
number of heterogeneous clients with different resources, network transcoders 
can be used to help achieve end-to-end system efficiency and scalability. 
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In hybrid wired/wireless networks, it is often necessary to 
simultaneously stream video to fixed clients on a wired network and to mobile 
clients on a wireless network. In such a hybrid system, it may often be 
5 desirable to send a full-bandwidth, high-resolution video stream to the fixed 
wired client, and a lower-bandwidth, medium-resolution video stream to the 
mobile wireless receiver. Conventional video streaming approaches, 
however, do not achieve the efficiency, security, and scalability necessary to 
readily accommodate the video streaming corresponding to hybrid 
10 wired/wireless networks. 

Yet another example of the drawbacks associated with conventional 
video streaming approaches is demonstrated in conjunction with wireless 
appliance networks. In many wireless appliance networks, mobile senders 

15 and receivers communicate with one another over wireless links. A sender's 
coverage area is limited by the power of the transmitted signal. Relay devices 
can be used to extend the wireless coverage area when intended receivers 
are beyond the immediate coverage area of the sender. However, in the case 
of heterogeneous clients within the same wireless network, it may be desired 

20 to provide a higher bandwidth, high-resolution video stream to the high power 
wireless receivers, and a lower bandwidth, low-resolution video stream to the 
low power wireless receivers. Once again, conventional video streaming 
approaches do not achieve the efficiency, security, and scalability necessary 
to readily accommodate such video streaming demands in wireless appliance 

25 networks. 

Although the above-listed discussion specifically mentions the 
shortcomings of prior art approaches with respect to the streaming of video 
data, such shortcomings are not limited solely to the streaming of video data. 
30 Instead, the problems of the prior art span various types of media including, 
but not limited to, audio-based data, image-based data, graphic data, web 
page-based data, and the like. 

Accordingly, what is needed is a method and/or system that can allow 
35 media data to be streamed in a secure and computationally efficient manner. 
What is also needed is a method and/or system that can satisfy the above 
need and that can also allow media data to be streamed to heterogeneous 
clients ("receiving nodes") that may have different display, power, 
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communication and computational capabilities and characteristics. The 
present invention provides a novel solution to these needs. 
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DISCLOSURE OF THE INVENTION 

The present invention provides, in one embodiment, a secure and 
computationally efficient method and system allowing media data to be 
streamed to a variety of receiving nodes having different capabilities and 
5 characteristics. In another embodiment, the present invention provides a 
secure and computationally efficient method and system for allowing media 
data to be streamed according to attributes of the communication channel. 
These and other technical advantages of the present invention will no doubt 
become obvious to those of ordinary skill in the art after having read the 
10 following detailed description of the preferred embodiments that are illustrated 
in the various drawing figures. 

The present invention pertains to a device and method thereof for 
packetizing scalably encoded and progressively encrypted data. The data 
15 can be any type of media data including video data, audio data, image data, 
graphic data, and web page data. 

The packetizing device includes a receiver adapted to receive a stream 
of data from an encoding and encrypting device, in which some or all of the 

20 data are scalably encoded and progressively encrypted. The device also 
includes a packetizer adapted to packetize some or all of the data into secure 
and scalable data packets. In one embodiment, the device includes a memory 
unit for storing the data received from the encoding and encrypting device 
prior to packetization of the data. In another embodiment, the device includes 

25 a memory unit for storing the secure and scalable data packets. In yet another 
embodiment, the device includes a transmitter for transmitting some or all of 
the data packets to a downstream device. 

The packetizing device can receive data from a storage unit of the 
30 encoding and encrypting device, or the packetizing device can receive data in 
real time as the data are output from the encoding and encrypting device. The 
data may include header data and payload data. The header data includes 
information corresponding to the payload data, and may be encrypted or 
unencrypted. The information in the header data allows a transcoder to 
35 transcode secure and scalable data packets without having to decrypt and 
decode those data packets. 

The packetizing device may receive all or a subset of the data 
processed by the encoding and encrypting device. Similarly, the packetizing 
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device may packetize all or a subset of the encoded and encrypted data. 
Likewise, the packetizing device may transmit all or a subset of the secure and 
scalable data packets. In each of these cases, the amount of data processed 
by the packetizing device depends on, for example, the attributes and 
5 capabilities of downstream devices and channels. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The accompanying drawings, which are incorporated in and form a part 
of this specification, illustrate embodiments of the invention and, together with 
the description, serve to explain the principles of the invention. 

5 

PRIOR ART FIGURE 1 a block diagram which illustrates the order in 
which conventional application-level encryption is performed. 

PRIOR ART FIGURE 2 is a block diagram which illustrates another 
10 conventional secure streaming system using network-level encryption. 

PRIOR ART FIGURE 3 is block diagram illustrating a conventional 
transcoding method. 

15 PRIOR ART FIGURE 4 is block diagram illustrating another conventional 

transcoding method. 

FIGURE 5 is a schematic diagram of an exemplary computer system 
used to perform steps of the present method in accordance with various 
20 embodiments of the present claimed invention. 

FIGURE 6 is a flowchart of steps performed in a secure and scalable 
encoding method in accordance with one embodiment of the present claimed 
invention. 

25 

FIGURE 7 is a block diagram of an encoding system in accordance with 
one embodiment of the present claimed invention. 

FIGURE 8 is a block diagram of an encoding system having a video 
30 prediction unit coupled thereto in accordance with one embodiment of the 
present claimed invention. 

FIGURE 9 is a block diagram of an encoding system having a video 
prediction unit integral therewith in accordance with one embodiment of the 
35 present claimed invention. 

FIGURE 1 0A is a schematic depiction of a frame of video data in 
accordance with one embodiment of the present claimed invention. 
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FIGURE 1 0B is a schematic depiction of the frame of video data of 
FIGURE 10A after segmentation into corresponding regions in accordance 
with one embodiment of the present claimed invention. 

5 FIGURE 1 0C is a schematic depiction of the frame of video data of 

FIGURE 10A after segmentation into corresponding non-rectangular regions 
in accordance with one embodiment of the present claimed invention. 

FIGURE 1 0D is a schematic depiction of the frame of video data of 
10 FIGURE 10A after segmentation into corresponding overlapping non- 
rectangular regions in accordance with one embodiment of the present 
claimed invention. 

FIGURE 1 1 is a flowchart of a process for decoding data which has 
15 been securely and scalably encoded in accordance with one embodiment of 
the present claimed invention. 

FIGURE 12 is a block diagram of a decoding system in accordance with 
one embodiment of the present claimed invention. 

20 

FIGURE 13 is a block diagram of a decoding system having a video 
prediction unit coupled thereto in accordance with one embodiment of the 
present claimed invention. 

25 FIGURE 14 is a block diagram of a decoding system having a video 

prediction unit integral therewith in accordance with one embodiment of the 
present claimed invention. 

FIGURE 15A is a block diagram of an exemplary hybrid wired/wireless 
30 network upon which embodiments of the present invention may be practiced. 

FIGURE 15B is a block diagram of an exemplary wireless network upon 
which embodiments of the present invention may be practiced. 

35 FIGURE 16 is a block diagram of a source node, an intermediate 

(transcoder) node, and a receiving node in accordance with one embodiment 
of the present invention. 
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FIGURE 17 is a block diagram of one embodiment of a transcoder 
device upon which embodiments of the present invention may be practiced in 
accordance with one embodiment of the present claimed invention. 

FIGURES 18A, 18B, 18C, 18D and 18E are data flow diagrams 
illustrating various embodiments of a method for transcoding data packets in 
accordance with one embodiment of the present claimed invention. 

FIGURE 19 is a flowchart of a process for transcoding data packets in 
accordance with one embodiment of the present claimed invention. 

FIGURE 20 is a schematic representation of a data packet including 
header data and scalably encoded, progressively encrypted data in 
accordance with one embodiment of the present claimed invention. 

FIGURE 21 is a schematic representation of a data packet including 
scalably encoded, progressively encrypted data in accordance with one 
embodiment of the present claimed invention. 

FIGURES 22A and 22B are block diagrams of devices for encoding and 
encrypting data in accordance with various embodiments of the present 
claimed invention. 

FIGURE 23 is a flowchart of the steps in a process for encoding and 
encrypting data in accordance with one embodiment of the present claimed 
invention. 

FIGURE 24 is a flowchart of the steps in a process for packetizing data 
in accordance with one embodiment of the present claimed invention. 

FIGURES 25A, 25B, 25C and 25D are block diagrams of devices for 
packetizing data in accordance with various embodiments of the present 
claimed invention. 

The drawings referred to in this description should be understood as 
not being drawn to scale except if specifically noted. 
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BEST MODES FOR CARRYING OUT THE INVENTION 

Reference will now be made in detail to the preferred embodiments of 
the invention, examples of which are illustrated in the accompanying 
drawings. While the invention will be described in conjunction with the 
5 preferred embodiments, it will be understood that they are not intended to limit 
the invention to these embodiments. On the contrary, the invention is intended 
to cover alternatives, modifications and equivalents, which may be included 
within the spirit and scope of the invention as defined by the appended claims. 
Furthermore, in the following detailed description of the present invention, 

10 numerous specific details are set forth in order to provide a thorough 

understanding of the present invention. However, it will be obvious to one of 
ordinary skill in the art that the present invention may be practiced without 
these specific details. In other instances, well known methods, procedures, 
components, and circuits have not been described in detail as not to 

15 unnecessarily obscure aspects of the present invention. 

It should be borne in mind, however, that all of these and similar terms 
are to be associated with the appropriate physical quantities and are merely 
convenient labels applied to these quantities. Unless specifically stated 

20 otherwise as apparent from the following discussions, it is appreciated that 
throughout the present invention, discussions utilizing terms such as 
"receiving," "segmenting," "encoding," "encrypting," "storing," "sending," 
"generating," "providing," "packetizing" or the like, refer to the actions and 
processes of a computer system, or similar electronic computing device. The 

25 computer system or similar electronic computing device manipulates and 
transforms data represented as physical (electronic) quantities within the 
computer system's registers and memories into other data similarly 
represented as physical quantities within the computer system memories or 
registers or other such information storage, transmission, or display devices. 

30 The present invention is also well suited to the use of other computer systems 
such as, for example, optical and mechanical computers. 

COMPUTER SYSTEM ENVIRONMENT OF THE 
PRESENT SECURE SCALABLE STREAMING INVENTION 

35 With reference now to Figure 5, portions of the present invention 

method and system are comprised of computer-readable and computer- 
executable instructions which reside, for example, in computer-usable media 
of a computer system. Figure 5 illustrates an exemplary computer system 500 
used in accordance with one embodiment of the present secure scalable 

40 streaming invention. It is appreciated that system 500 of Figure 5 is exemplary 
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only and that the present invention can operate on or within a number of 
different computer systems including general purpose networked computer 
systems, embedded computer systems, routers, switches, server devices, 
client devices, various intermediate devices/nodes, stand alone computer 
5 systems, and the like. Additionally, computer system 500 of Figure 5 is well 
adapted to having computer readable media such as a floppy disk, a compact 
disc, and the like coupled thereto. Such computer readable media are not 
shown coupled to computer system 500 in Figure 5 for purposes of clarity. 

10 System 500 of Figure 5 includes an address/data bus 502 for 

communicating information, and a central processor unit 504 coupled to bus 
502 for processing information and instructions. System 500 also includes 
data storage features such as a computer usable volatile memory 506, e.g. 
random access memory (RAM), coupled to bus 502 for storing information and 

15 instructions for central processor unit 504, computer usable non-volatile 
memory 508 (e.g. read only memory, ROM) coupled to bus 502 for storing 
static information and instructions for the central processor unit 504, and a 
data storage unit 510 (e.g., a magnetic or optical disk and disk drive) coupled 
to bus 502 for storing information and instructions. System 500 of the present 

20 invention also includes an optional alphanumeric input device 512 including 
alphanumeric and function keys coupled to bus 502 for communicating 
information and command selections to central processor unit 504. System 
500 also optionally includes an optional cursor control device 514 coupled to 
bus 502 for communicating user input information and command selections to 

25 central processor unit 504. System 500 of the present embodiment also 
includes an optional display device 516 coupled to bus 502 for displaying 
information. 

Referring still to Figure 5, optional display device 516 may be a liquid 
30 crystal device, cathode ray tube, or other display device suitable for creating 
graphic images and alphanumeric characters recognizable to a user. 
Optional cursor control device 514 allows the computer user to dynamically 
signal the two dimensional movement of a visible symbol (cursor) on a display 
screen of display device 516. Many implementations of cursor control device 
35 514 are known in the art including a trackball, mouse, touch pad, joystick or 
special keys on alphanumeric input device 512 capable of signaling 
movement of a given direction or manner of displacement. Alternatively, it will 
be appreciated that a cursor can be directed and/or activated via input from 
alphanumeric input device 512 using special keys and key sequence 
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commands. The present invention is also well suited to directing a cursor by 
other means such as, for example, voice commands. A more detailed 
discussion of the present secure scalable streaming invention is found below. 

5 GENERAL DESCRIPTION OF THE PRESENT 

SECURE SCALABLE STREAMING INVENTION 

With reference next to Figure 6, Figure 1 1 , and Figure 19, flowcharts 

600, 1100 and 1900, respectively, illustrate exemplary steps used by the 

various embodiments of the present invention. Flowcharts 600, 1 100 and 

10 1900 include processes of the present invention which, in one embodiment, 
are carried out by a processor under the control of computer-readable and 
computer-executable instructions. The computer-readable and computer- 
executable instructions reside, for example, in data storage features such as 
computer usable volatile memory 506, computer usable non-volatile memory 

15 508, and/or data storage device 510 of Figure 5. The computer-readable and 
computer-executable instructions are used to control or operate in conjunction 
with, for example, central processing unit 504 of Figure 5. 

As an overview, the present invention is directed towards any data 

20 which can be scalably encoded and, specifically, any data that combine 

scalable encoding with progressive encryption. For purposes of the present 
application, scalable coding is defined as a process which takes original data 
as input and creates scalably coded data as output, where the scalably coded 
data have the property that portions of it can be used to reconstruct the original 

25 data with various quality levels. Specifically, the scalably coded data are often 
thought of as an embedded bitstream. The first portion of the bitstream can be 
used to decode a baseline-quality reconstruction of the original data, without 
requiring any information from the remainder of the bitstream, and 
progressively larger portions of the bitstream can be used to decode improved 

30 reconstructions of the original data. For purposes of the present application, 
progressive encryption is defined as a process which takes original data (plain 
text) as input and creates progressively encrypted data (cipher text) as output, 
where the progressively encrypted data have the property that the first portion 
can be decrypted alone, without requiring information from the remainder of 

35 the original data; and progressively larger portions can be decrypted with this 
same property, in which decryption can require data from earlier but not later 
portions of the bitstream. 
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ENCODING METHOD AND SYSTEM 
Although specific steps are disclosed in flowchart 600 of Figure 6, such 
steps are exemplary. That is, the present invention is well suited to performing 
various other steps or variations of the steps recited in Figure 6. Additionally, 
5 for purposes of clarity and brevity, the following discussion and examples will 
specifically deal with video data. The present invention, however, is not 
limited solely to use with video data. Instead, the present invention is well 
suited to use with audio-based data, image-based data, web page-based 
data, graphic data and the like ("media data"). Specifically, the present 
10 invention is directed towards any data in which scalable coding is combined 
with progressive encryption. In step 602 of Figure 6, in one embodiment, the 
present invention recites receiving video data. In one embodiment, the video 
data are comprised of a stream of uncompressed video frames which are 
received by segmenter 702 of the encoder system 700 of Figure 7. 

15 

In another embodiment of the present invention, the video data are 
comprised of prediction error video data generated by a video prediction unit 
(VPU). As shown Figure 8, in one embodiment of the present invention 
encoder system 700 has a VPU 800 coupled thereto. VPU 800 generates and 
20 forwards prediction error video data to segmenter 702 of encoder system 700. 
Although VPU 800 of Figure 8 is disposed outside of encoding system 700, 
the present invention is also well suited to having VPU 800 integral with 
encoding system 700. Figure 9 illustrates one embodiment of the present 
invention in which VPU 800 is integral with encoding system 700. 

25 

With reference now to step 604 of Figure 6, the present embodiment of 
the present invention then segments the received video data into 
corresponding regions. Figure 10A provides a schematic depiction of a video 
frame 1000. Video data corresponding to video frame 1000 are received by 

30 segmenter 702 of Figures 7, 8, and 9. Figure 10B depicts the same video 
frame 1000 after segmenter 702 has segmented video frame 1000 into 
corresponding regions 1002, 1004, 1006, 1008, 1010, and 1012. Although 
such a quantity and configuration of regions is shown in Figure 10B, such a 
tiling quantity and configuration is intended to be exemplary only. As one 

35 example, Figure 10C illustrates another example of segmentation in which 
segmenter 702 has segmented video frame 100 into various non-rectangular 
regions 1014, 1016, 1018, 1020, and 1022. As another example, Figure 10D 
illustrates another example of segmentation in which segmenter 702 has 
segmented video frame 100 into various non-rectangular and overlapping 
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regions 1024, 1026, 1028, 1030, and 1032. The overlapping portions are 
denoted by dotted lines. The present invention is also well suited to an 
approach in which segmenter 702 has various rectangular regions configured 
in an overlapping arrangement. Furthermore, the present invention is also 
5 well suited to an embodiment in which the regions change from frame to 
frame. Such an embodiment is employed, for example, to track a foreground 
person as they move. 

Referring now to step 606, encoder 704 of Figures 7, 8 and 9 then 

10 scalably encodes the regions into scalable video data. For purposes of the 
present application, scalable coding is defined as a process which takes 
original data as input and creates scalably coded data as output, where the 
scalably coded data have the property that portions of it can be used to 
reconstruct the original data with various quality levels. Specifically, the 

15 scalably coded data are often thought of as an embedded bitstream. The first 
portion of the bitstream can be used to decode a baseline-quality 
reconstruction of the original data, without requiring any information from the 
remainder of the bitstream, and progressively larger portions of the bitstream 
can be used to decode improved reconstructions of the original data. That is, 

20 a separate region or regions of a video frame are encoded into one or more 
data packets. The scalable video data generated by the present embodiment 
have the property that a first small portion of the data can be decoded into 
baseline quality video, and larger portions can be decoded into improved 
quality video. It is this property that allows data packets to be transcoded to 

25 lower bitrates or spatial resolutions simply by truncating the data packet. This 
process of truncation will be discussed in further detail below. 

With reference still to step 606, in one embodiment of the present 
invention, each region is coded by encoder 704 into two portions: header 

30 data and scalable video data. Hence, in such an embodiment, each data 
packet contains header data and scalable video data. The header data 
describe, for example, the region (e.g., the location of the region within the 
video frame) that the data packet represents and other information used for 
subsequent transcoding and decoding operations in accordance with the 

35 present invention. Furthermore, in one embodiment, the header data contain 
information including a series of recommended truncation points for data 
packet transcoders. The scalable video data contain the actual coded video. 
In the case of intraframe coding, the video data may be the coded pixels; while 
in the case of interframe coding, it may be the motion vectors and coded 
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residuals that result from motion-compensated prediction. In the present 
embodiments, scalable coding techniques are used in both cases to create an 
embedded or scalable data packet that can be truncated to lower the 
resolution or fidelity of the coded video data. In still another embodiment of 
5 the present invention, the scalably encoded video data are prepared by 
encoder 704 without corresponding header data. 

As recited in step 608, the present embodiment then progressively 
encrypts the scalable video data to generate progressively encrypted scalable 

10 video data. That is, packetizer and encrypter 706 of Figures 7, 8, and 9 

employ progressive encryption techniques to encrypt the scalable video data. 
For purposes of the present application, progressive encryption is defined as a 
process which takes original data (plain text) as input and creates 
progressively encrypted data (cipher text) as output, where the progressively 

15 encrypted data have the property that the first portion can be decrypted alone, 
without requiring information from the remainder of the original data; and 
progressively larger portions can be decrypted with this same property, in 
which decryption can require data from earlier but not later portions of the 
bitstream. Progressive encryption techniques include, for example, cipher 

20 block chains or stream ciphers. These progressive encryption methods have 
the property that the first portion of the data is encrypted independently, then 
later portions are encrypted based on earlier portions. When properly 
matched with scalable coding and packetization, progressive encryption 
preserves the ability to transcode data packets with simple data packet 

25 truncation. More specifically, progressive encryption methods have the 

property that smaller blocks of data are encrypted progressively. While block 
code encryption with small block sizes is not very secure, progressive 
encryption methods add a degree of security by feeding encrypted data of 
earlier blocks into the encryption of a later block. Decryption can then be 

30 performed progressively as well. In one embodiment, the first small block of 
cipher text is decrypted into plain text by itself while later blocks of cipher text 
depend on the decrypted plain text from earlier blocks. Thus, earlier blocks of 
cipher text can be decrypted without knowledge of the entire cipher text 
segment. This progressive nature of cipher block chains and stream ciphers 

35 matches nicely with the progressive or embedded nature of scalable coding. 
Although encoding system 700 depicts a combined packetizer and encrypter 
module 706, such a depiction is exemplary only, as encoding system 700 of 
the present invention is well suited to having separate and distinct packetizer 
and encrypter modules. 

16 
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In prior art approaches, entire data packets were encrypted with one 
long block code. As a result, decryption was not possible unless the data 
packet was received in its entirety. However, the present invention is using 
5 scalable data packets and it is desired to transcode the stream of scalable 
data packets by data packet truncation. Therefore, the present invention 
encrypts the data packets in a similarly progressive manner. Hence, unlike 
conventional approaches, the present invention is data packet loss resilient. 
That is, should a data packet be lost, decryption of the remaining data packets 
10 is not further complicated and is still readily achievable. This combination of 
scalable encoding and progressive encryption enables the advantageous 
transcoding operations described in detail below. 

With reference still to step 608, in one embodiment of the present 
15 invention, while the payload data (e.g., the scalable video data) are encrypted 
progressively, the header data are left unencrypted so that transcoding nodes 
can use this information to make transcoding decisions. For example, in one 
embodiment, the unencrypted header contains information such as 
recommended truncation points within the encrypted payload data. In another 
20 embodiment, these header data are used to achieve near rate distortion (RD)- 
optimal bitrate reduction by intermediate transcoding nodes. Moreover, in the 
present embodiment, the transcoding nodes can use the header data to make 
transcoding decisions without requiring decryption of the progressively 
encrypted scalable video data or the header data. In yet another embodiment 
25 of the present invention, the header data are encrypted to add additional 
security. 

Referring now to step 610, the present invention then packetizes the 
progressively encrypted scalable video data. In one embodiment, a 

30 packetizer and encrypter 706 of Figures 7, 8, and 9 combines and packetizes 
the unencrypted header data with the progressively encrypted scalable video 
data. The resulting secure scalable data packets are then available to be 
streamed to desired receivers. In another embodiment, packetizer and 
encrypter 706 packetizes the progressively encrypted scalable video data and 

35 the encrypted header data. Furthermore, in an embodiment which does not 
include header data, packetizer and encrypter 706 packetizes only the 
progressively encrypted scalable video data. 
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Encoding system 700 securely and scalabiy encodes video data. More 
specifically, encoding system 700 combines scalable coding with progressive 
encryption techniques. The resulting scalabiy encoded, progressively 
encrypted, and packetized video streams have the feature that subsequent 
5 transcoding operations such as bitrate reduction and spatial downsampling 
can be performed (via data packet truncation or data packet elimination, for 
example) without decrypting the packetized data and thus while maintaining 
the security of the system. The present invention is also well suited to an 
embodiment in which only some, but not all, of the regions formed by 
10 segmenter 702 are ultimately forwarded from encoding system 700. As an 
example, in one embodiment, the foreground of a video data image is 
forwarded, as the background image may not have changed since a previous 
transmission, or perhaps the background image does not contain data of 
interest. 

15 

DECODING METHOD AND SYSTEM 
Although specific steps are disclosed in flowchart 1100 of Figure 11, 
such steps are exemplary. That is, the present invention is well suited to 
performing various other steps or variations of the steps recited in Figure 1 1 . 

20 In step 1 102 of Figure 1 1 , the present invention receives a data packet 

containing progressively encrypted and scalabiy encoded video data. More 
specifically, decrypter 1202 of decoding system 1200 (Figure 12) receives the 
data packet containing progressively encrypted and scalabiy encoded video 
data. In one embodiment, the received data packet also includes header data 

25 wherein the header data provide information corresponding to the scalabiy 
encoded video data. In yet another embodiment, the received data packet 
also includes encrypted header data providing information corresponding to 
the scalabiy encoded video data. 

30 As recited in step 1 104, the present invention then decrypts the data 

packet containing the progressively encrypted and scalabiy encoded video 
data to generate scalabiy encoded regions. That is, decrypter 1202 of Figure 
12 decrypts the progressively encrypted and scalabiy encoded video data to 
generate scalabiy encoded regions. Furthermore, in an embodiment in which 

35 the received data packet includes encrypted header data, decrypter 1202 also 
decrypts the encrypted header data. 

Referring now to step 1 106, the present embodiment then decodes the 
scalabiy encoded regions to provide decoded regions. As described above in 

18 



HP10018127/JPW/WAZ 



conjunction with the description of encoding system 700 of Figures 7, 8, and 9, 
a video frame 1000 as shown in Figure 10A can be segmented in multiple 
corresponding regions 1002, 1004, 1006, 1008, 1010, and 1012 as shown in 
Figure 10B. 

5 

At step 1 108, the present invention then assembles the decoded 
regions to provide video data. Moreover, assembler 1206 of decoding system 
1200 of Figure 12 assembles the decoded regions to provide video data. In 
one embodiment of the present invention, decoding system 1200 then 

10 provides, as output, video data in the form of an uncompressed video stream. 
In another embodiment of the present invention, assembler 1206 outputs 
video data comprised of prediction error video data suitable for use by a video 
prediction unit (VPU). As shown Figure 13, in one embodiment of the present 
invention, decoder system 1200 has a VPU 1300 coupled thereto. VPU 1300 

15 uses the output of assembler 1206 to ultimately provide an uncompressed 
stream of video frame data. Although VPU 1300 of Figure 13 is disposed 
outside of decoding system 1200, the present invention is also well suited to 
having VPU 1300 integral with decoding system 1200. Figure 14 illustrates 
one embodiment of the present invention in which VPU 1300 is integral with 

20 decoding system 1200. Hence, the present invention provides a method and 
system for decoding video data which has been securely and scalably 
encoded. 



TRANSCODING METHOD AND SYSTEM 
25 Figure 15A is a block diagram of an exemplary hybrid wired/wireless 

network 1500 upon which embodiments of the present invention may be 
practiced. In hybrid wired/wireless network 1500, media (e.g., video) data are 
streamed to fixed clients (stationary receiving nodes) via a wired link and to 
mobile clients (moving receiving nodes) via a wireless link. 

30 

In the present embodiment, hybrid wired/wireless network 1500 
includes a wired sender (source 1510), a wired high-resolution receiver 1520, 
and a wireless medium-resolution receiver 1540. In this system, source 1510 
generates a full-bandwidth, high-resolution video stream 1550a that is sent to 
35 high-resolution receiver 1520. A transcoder 1530, placed either at source 

1510, at medium-resolution receiver 1540, or at an intermediate node such as 
a wired/wireless gateway, transcodes the stream 1550a into a lower- 
bandwidth, medium-resolution video stream 1550b which is then sent to 
medium-resolution receiver 1540. 
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Figure 15B is a block diagram of an exemplary wireless network 1501 
(e.g., a wireless appliance network) upon which embodiments of the present 
invention may be practiced. In wireless appliance networks, mobile senders 
5 and receivers communicate with one another over wireless links. A sender's 
coverage area is limited by the power of the transmitted signal. Relay devices 
can be used to extend the wireless coverage area when intended receivers 
are beyond the immediate coverage area of the sender. In the case of 
heterogeneous receivers (e.g., receiving nodes having different display, 
10 power, computational, and communication characteristics and capabilities), 
transcoders can be used to adapt a video stream for a particular receiver or 
communication link. Transcoding can be performed in a relay device or in a 
receiver which also acts as a relay. Transcoding can also be performed by the 
sender or by the receiving node. 

15 

In the present embodiment, wireless network 1501 includes a wireless 
sender (source 1510), a high-resolution receiver and transcoder 1560, and a 
medium-resolution (lower bandwidth) receiver 1540. In wireless network 
1501, the high-resolution receiver 1560 receives and transcodes the high- 
20 resolution video stream 1550a, and relays the resulting lower-bandwidth 
stream 1550b to the medium-resolution receiver 1540. 

Referring to Figures 15A and 15B, both hybrid wired/wireless network 
1500 and wireless network 1501 use network transcoders to transcode video 

25 streams 1550a into lower bandwidth streams 1550b that match the display 
capabilities of the target wireless nodes (e.g., medium-resolution receiver 
1540). Generally speaking, these networks illustrate how network transcoding 
can enable efficient use of wireless spectrum and receiver resources by 
transcoding media (e.g., video) streams into formats better suited for 

30 transmission over particular channels and for the capabilities of the receiving 
nodes. 

Figure 16 is a block diagram of a system 1600 including a source node 
1610, an intermediate (transcoder) node 1620, and a receiving node 1630 in 
35 accordance with one embodiment of the present invention. In this 

embodiment, transcoder 1620 is a separate node transposed between source 
node 1610 and receiving node 1630. However, the functions performed by 
transcoder 1620 may instead be performed by source node 1610 or by 
receiving node 1630. 
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In the present embodiment, source node 1610 encodes and/or encrypts 
a stream of data packets and sends these data packets to transcoder 1 620, as 
described above, in one embodiment, each of the data packets in the stream 
5 has a header portion and a payload portion (see Figure 20, below); in another 
embodiment, the data packet has only a payload portion (see Figure 21 , 
below). The payload portion carries the data, while the header portion carries 
information that is used by transcoder 1620 to transcode the payload portion. 
A data packet, including the information carried by the header portion, and the 
10 transcoding method used by transcoder 1620 are further described below. In 
one embodiment, only the payload portion is encrypted and encoded. In 
another embodiment, the payload portion is encrypted and encoded, and the 
header portion is also encrypted. 

15 In the present embodiment, transcoder 1620 performs a transcoding 

function on the data packets received from source node 1610. The 
transcoding function performed by transcoder 1620 is described in 
conjunction with Figure 19, below. The purpose of the transcoding function is 
to configure the stream of data packets according to the attributes downstream 

20 of transcoder 1620, such as the attributes of the receiving node 1630 or the 
attributes of communication channel 1625 linking transcoder 1620 and 
receiving node 1630. The transcoding function can include, for example, 
truncation of the data packets or elimination of certain data packets from the 
stream. In the case in which the stream is already configured for the receiving 

25 node 1630 or for communication channel 1625, the transcoding function 
consists of a pass-through of the data packets in the stream without 
modification. 

Of particular significance, in accordance with the present invention, 
30 transcoder 1620 performs a transcoding function without decrypting and/or 
decoding the data packets (specifically, the media data in the data packets). 
In the embodiment in which the data packets have a header portion and a 
payload portion, and where the header portion is encrypted, transcoder 1620 
only decrypts the header portion. In either case, in comparison to a 
35 conventional transcoder, transcoder 1620 of the present invention requires 
less computational resources because there is no need to decrypt the media 
data. In addition, the present invention provides end-to-end security while 
enabling very low complexity transcoding to be performed at intermediate, 
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possibly untrusted, nodes without compromising the security of the media 
data. 

Continuing with reference to Figure 16, transcoder 1620 has 
5 knowledge of the attributes of receiving node 1630 and/or communication 
channel 1625. These attributes include, but are not limited to, the display, 
power, communication and computational capabilities and characteristics of 
receiving node 1630 or the available bandwidth on communication channel 
1625. For example, in one embodiment, transcoder 1620 receives the 
10 attribute information from receiving node 1630, or transcoder 1620 reads this 
information from receiving node 1630. In another embodiment, transcoder 
1620 may be implemented as a router in a network; the router can determine if 
there is congestion on the next "hop" and transcode the stream of data packets 
accordingly. 

15 

In the present embodiment, after transcoding, transcoder 1620 sends 
the resultant stream of data packets, comprising the encoded and encrypted 
data packets, to receiving node 1 630. 

20 Figure 17 is a block diagram of one embodiment of a transcoder device 

1620 upon which embodiments of the present invention may be practiced. In 
this embodiment, transcoder 1620 includes a receiver 1710 and a transmitter 
1720 for receiving a stream of data packets from source node 1610 (Figure 
16) and for sending a stream of data packets to receiving node 1630 (Figure 

25 16), respectively. Receiver 1710 and transmitter 1720 are capable of either 
wired or wireless communication. Separate receivers and transmitters, one 
for wired communication and one for wireless communication, may also be 
used. It is appreciated that receiver 1710 and transmitter 1720 may be 
integrated as a single device (e.g., a transceiver). 

30 

Continuing with reference to Figure 17, transcoder device 1620 may 
include an optional controller 1730 (e.g., a processor or microprocessor), an 
optional decrypter 1740, and an optional memory 1750, or a combination 
thereof. In one embodiment, decrypter 1740 is used to decrypt header 
35 information. In another embodiment, memory 1750 is used to accumulate 
data packets received from source node 1610 before they are forwarded to 
receiving node 1630 (Figure 16). 
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Figures 18A, 18B, 18C, 18D and 18E are data flow diagrams illustrating 
various embodiments of a method for transcoding data packets in accordance 
with the present invention. In the embodiments of Figures 18A-18D, the data 
packets each have a header portion and a payload portion; in the embodiment 
5 of Figure 18E, the data packets do not have a header portion. In each of the 
embodiments of Figures 18A-18E, the data packets (specifically, the media 
data) are encrypted and may be encoded. The embodiments of Figures 18A- 
18E are separately described in order to more clearly describe certain aspects 
of the present invention; however, it is appreciated that the present invention 
10 may be implemented by combining elements of these embodiments. 

In accordance with the present invention, the method for transcoding 
data packets is performed on the encrypted data packets; that is, the media 
data are not decrypted. Transcoding functions can include truncation of the 
15 data packets (specifically, the payload portions of the data packets), 

eliminating certain data packets from the stream, or passing the data packets 
through without modification. 

With reference first to Figure 18A, incoming encrypted and/or encoded 
20 data packets are received by transcoder 1620. In this embodiment, the 

header portion of each data packet is not encrypted. Transcoder 1620 reads 
the header portion, which contains information that can be used to make 
transcoding decisions. In one embodiment, the information in the header 
portion includes specification of the truncation points. In another embodiment, 
25 the truncation points are derived from the information provided in the header. 

For example, the header portion may contain information specifying 
recommended points (e.g., a number of a bit) for truncating the payload 
portion of the data packets. It is appreciated that each data packet may have a 

30 different truncation point. The recommended truncation point can be selected 
using a variety of techniques. In one embodiment, the truncation point for 
each data packet is specified according to an analysis such as a rate- 
distortion (RD) analysis, so that the stream of data packets can be compressed 
to a rate that is RD optimal or near-RD optimal. In another embodiment, the 

35 header portion contains information that describes the RD curves generated 
by the RD analysis, and the truncation points are derived from further analysis 
of the RD curves. 
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In the present embodiment, RD optimal coding is achieved by 
generating an RD plot for each region of a video image, and then operating on 
all regions at the same slope that generates the desired total bitrate. Near- 
optimal transcoding can be achieved at the data packet level by placing the 
5 optimal RD cutoff points for a number of quality levels in the header portions of 
the data packets. Then, transcoder 1620 (Figure 16) can truncate each packet 
at the appropriate cutoff point; thus, the resulting packets will contain the 
appropriate number of bits for each region of the image for the desired quality 
level. Transcoder 1620 reads each paqket header, then truncates the packet 

10 at the appropriate point. For example, if three regions in an image are coded 
into separate packets, for each region three RD optimal truncation points are 
identified and their locations placed in the respective packet header. 
Transcoder 1620 can choose to operate at any of the three RD points (or 
points in between), and then can truncate each packet at the appropriate cutoff 

15 point. 

The header portion may also contain information identifying each data 
packet by number, for example. Accordingly, transcoder 1620 can eliminate 
certain data packets from the stream; for example, if every other packet is to be 
20 eliminated (e.g., the odd-numbered packets), transcoder 1620 can use the 
header information to identify the odd-numbered data packets and eliminate 
those from the stream of data packets. 

The embodiment of Figure 18B is similar to that of Figure 18A, except 
25 that the header portion of each data packet is encrypted. In this case, 

transcoder 1620 first decrypts the header portion before reading the header 
information and operating on the stream of data packets as described above. 

In the embodiment of Figure 18C, data packets are accumulated in 
30 memory. That is, instead of a first-in/first-out type of approach, a subset of the 
data packets in the stream is accumulated and stored in memory (e.g., 
memory 1750 of Figure 17) before they are forwarded to the receiving node. 
In this embodiment, the header information for all of the accumulated data 
packets in the subset is used to make transcoding decisions. The transcoding 
35 decisions are made based on the attributes of the receiving node 1630 or the 
attributes of the communication channel 1625 (Figure 16), as described 
previously herein. It may be possible, and perhaps desirable, to configure the 
stream of data packets according to the attributes of the receiving node or 
communication channel without operating on every data packet in the stream. 
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For example, instead of truncating all of the data packets in the subset, a 
decision may be made to truncate only a portion of the packets in the subset, 
or to truncate the packets at a point other than the recommended truncation 
point. 

5 

In the embodiment of Figure 18D, transcoder 1620 receives information 
from the downstream receiving node (e.g., receiving node 1630 of Figure 16). 
In one embodiment, the information describes attributes of receiving node 
1630, such as its display, power, computational and communication 

10 capabilities and characteristics. Based on the information received from 

receiving node 1630, transcoder 1620 can make transcoding decisions based 
on the information in the header portions of the data packets. For example, 
transcoder 1620 can pick a truncation point depending on whether receiving 
node 1630 is a medium- or low-resolution device, and transcoder 1620 can 

15 choose not to modify the stream of data packets if receiving node 1 630 is a 
high-resolution device. Similarly, transcoder 1620 can receive information 
describing the attributes of communication channel 1625 (Figure 16). 

In the embodiment of Figure 18E, the incoming data packets do not 
20 have a header portion. Accordingly, transcoder 1620 makes transcoding 
decisions based on a pre-defined set of rules. That is, instead of truncating 
each data packet at a different point specified by the information in the header 
portion, transcoder 1620 may truncate all data packets in the stream at the 
same point, depending on the attributes of the receiving node or 
25 communication channel. 

Figure 19 is a flowchart of the steps in a process 1900 for transcoding 
data packets in accordance with one embodiment of the present invention. In 
one embodiment, process 1 900 is implemented by transcoder device 1 620 
30 (Figure 17) as computer-readable program instructions stored in memory 

1750 and executed by controller 1730. Although specific steps are disclosed 
in of Figure 19, such steps are exemplary. That is, the present invention is 
well suited to performing various other steps or variations of the steps recited 
in Figure 19. 



35 



In step 1910 of Figure 19, a stream of data packets is received from a 
source node (e.g., source 1610 of Figure 16). In the present embodiment, the 
data packets include encrypted data. In one embodiment, the data are also 
encoded. In another embodiment, the data packets include a header portion 
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and a payload portion. In one embodiment, the header portion is also 
encrypted. 

In step 1915 of Figure 19, in one embodiment, information describing 
5 the attributes of a downstream receiving node (e.g., receiving node 1630 of 
Figure 16) or communication channel (e.g., communication channel 1625 of 
Figure 16) is received. In another embodiment, the attributes of receiving 
node 1630 or communication channel 1625 are already known. 

10 In step 1920 of Figure 19, a transcoding function is performed on the 

stream of data packets to configure the stream according to the attributes of 
receiving node 1630. Significantly, the transcoding function is performed 
without decrypting the data in the data packets. In one embodiment, the 
transcoding function is performed on information provided by the header 

15 portion of each data packet. In one such embodiment, the header information 
provides recommended truncation points for the payload portion of the 
respective data packet. In another embodiment, the truncation points are 
derived from the information provided in the header portion. 

20 In step 1922, in one embodiment, the transcoding function eliminates 

certain data packets from the stream. In step 1924, in one embodiment, the 
transcoding function truncates the data in the data packets. It is appreciated 
that each data packet may have a different truncation point. In step 1926, in 
one embodiment, the transcoding function passes the data packets through 

25 without modification. 

In step 1930, the transcoded data packets (still encrypted and/or 
encoded) are sent to receiving node 1630. 

30 In summary, the above-listed embodiment of the present invention 

provides a secure method and system for transcoding data for a variety of 
downstream attributes, such as the attributes of receiving nodes having 
different capabilities and characteristics or the attributes of the communication 
between the transcoder and a receiving node. Because the encrypted data do 

35 not need to be decrypted and then encrypted again, the computational 

resources needed for transcoding the stream of data packets is significantly 
reduced, and the security of the data is not compromised. 
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SECURE SCALABLE DATA PACKET 
With reference now to Figure 20, a schematic representation of a data 
packet 2000 formed in accordance with one embodiment of the present 
invention is shown. Furthermore, as mentioned above, for purposes of clarity 
5 and brevity, the following discussion and examples will specifically deal with 
video data. The present invention, however, is not limited solely to use with 
video data. Instead, the present invention is well suited to use with audio- 
based data, image-based data, web page-based data, and the like. It will be 
understood that in the present embodiments, data packet 2000 is generated 

10 by encoding system 700 of Figures 7, 8, and 9, operated on by transcoder 
1620 of Figures 16, 18A, 18B, 18C, 18D, and 18E , and then ultimately 
forwarded to decoding system 1200 of Figures 12, 13, and 14. During the 
aforementioned process, data packet 2000 is stored on computer readable 
media residing in, and causes a functional change or directs the operation of, 

15 the devices (e.g., general purpose networked computer systems, embedded 
computer systems, routers, switches, server devices, client devices, various 
intermediate devices/nodes, stand alone computer systems, and the like) in 
which, for example, transcoder 1620 and/or decoder 1200 are implemented. 

20 In the embodiment of Figure 20, data packet 2000 includes header data 

portion 2002 and scalably encoded, progressively encrypted video data 
portion 2004. As mentioned above, header data portion 2002 includes 
information that is used by transcoder 1620 to transcode the scalably 
encoded, progressively encrypted video data portion 2004. For example, 

25 header data portion 2002 may contain information specifying recommended 
points (e.g., a number of a bit) for truncating the payload portion (e.g., the 
scalably encoded, progressively encrypted video data portion 2004) of data 
packet 2000. Header data portion 2002 may also contain information 
identifying each data packet by number, for example. Accordingly, transcoder 

30 1620 can eliminate certain data packets from the stream; for example, if every 
other packet is to be eliminated (e.g., the odd-numbered packets), transcoder 
1620 can use the information in header data portion 2002 to identify the odd- 
numbered data packets and eliminate those from the stream of data packets. 

35 With reference still to Figure 20, data packet 2000 also includes 

potential truncation points 2006, 2008, and 2010 within scalably encoded, 
progressively encrypted video data portion 2004. Although such truncation 
points are shown in Figure 20, the configuration of truncation points 2006, 
2008, and 2010 is exemplary only. That is, the present invention is well suited 
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to having a lesser or greater number of truncation points, and to having the 
truncation points located other than where shown in Figure 20. Again, as 
mentioned above, truncation points 2006, 2008, and 2010 are used by 
transcoder 1620 during its operation on packet 2000. Additionally, in one 
5 embodiment of the present invention, header data portion 2002 is encrypted. 

In the embodiment of Figure 21, data packet 2100 does not include a 
header data portion, and instead includes only scalably encoded, 
progressively encrypted video data portion 2104. With reference still to Figure 

10 21 , data packet 2100 also includes potential truncation points 2104, 2106, and 
2108 within scalably encoded, progressively encrypted video data portion 
2104. Although such truncation points are shown in Figure 21, the 
configuration of truncation points 2104, 2106, and 2108, is exemplary only. 
That is, the present invention is well suited to having a lesser or greater 

15 number of truncation points, and to having the truncation points located other 
than where shown in Figure 21. Again, as mentioned above, truncation points 
2104, 2106, and 2108 are used by transcoder 1620 during its operation on 
packet 2100. 

20 Thus, the present invention provides, in one embodiment, a secure and 

scalable encoding method and system for use in the streaming of data. The 
present invention further provides, in one embodiment, a method for decoding 
data which has been securely and scalably encoded. 

25 ENCODING AND ENCRYPTING DEVICES 

FOR SECURE SCALABLE DATA STREAMING 

Figure 22A is a block diagram of a device 2200 for scalably encoding 

and progressively encrypting data in accordance with one embodiment of the 

present claimed invention. As an overview, the present invention is directed 

30 towards any data which can be scalably encoded and, specifically, any data 
that combine scalable encoding with progressive encryption. For purposes of 
the present application, scalable coding is defined as a process which takes 
original data as input and creates scalably coded data as output, where the 
scalably coded data have the property that portions of it can be used to 

35 reconstruct the original data with various quality levels. Specifically, the 

scalably coded data are often thought of as an embedded bitstream. The first 
portion of the bitstream can be used to decode a baseline-quality 
reconstruction of the original data, without requiring any information from the 
remainder of the bitstream, and progressively larger portions of the bitstream 

40 can be used to decode improved reconstructions of the original data. For 
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purposes of the present application, progressive encryption is defined as a 
process which takes original data (plain text) as input and creates 
progressively encrypted data (cipher text) as output, where the progressively 
encrypted data have the property that the first portion can be decrypted alone, 
5 without requiring information from the remainder of the original data; and 
progressively larger portions can be decrypted with this same property, in 
which decryption can require data from earlier but not later portions of the 
bitstream. 

10 In the present embodiment, device 2200 includes a segmenter 2202 

coupled to an encoder 2204, which in turn is coupled to an encrypter 2206. 
The functionality of device 2200 is described in conjunction with Figure 23, 
below. 

15 Significantly, in this embodiment, device 2200 of Figure 22A does not 

include packetizing device 2208 as an integrated unit; instead, device 2200 is 
coupled to packetizing device 2208 disposed outside of device 2200. As 
such, different types of packetization methods can be used with device 2200, 
depending on the capabilities of downstream channels and devices, for 

20 example. In the present embodiment, packetizing device 2208 receives data 
from device 2200 in real time, that is, as the data are encoded and encrypted. 

Figure 22B is a block diagram of a device 2200a for scalably encoding 
and progressively encrypting data in accordance with another embodiment of 

25 the present claimed invention. In this embodiment, device 2200a includes a 
storage unit 2210 for storing encoded and encrypted data (specifically, 
scalably encoded and progressively encrypted data) that are output from 
encoder 2206. Thus, packetizing device 2208 can receive data from device 
2200a in real time as the data are encoded and encrypted, or at a later time 

30 packetizing device 2208 can receive data from device 2200a that are stored in 
storage unit 2210. In the latter case, packetizing device 2208 can receive all 
of or a selected portion of the data in storage unit 2210. Thus, for example, the 
data can be packetized for different types of channels (e.g., channels having 
different bandwidth), for different types of downstream devices (e.g., receiving 

35 nodes having different display, power, computational and communication 
characteristics and capabilities), or using different packetization methods. 
Additional information is provided in conjunction with Figure 24, below. 
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Figure 23 is a flowchart of the steps in a process 2300 for encoding and 
encrypting data in accordance with one embodiment of the present claimed 
invention. Although specific steps are illustrated in Figure 23, such steps are 
exemplary, and the present invention is well suited to performing various other 
5 steps or variations of the steps included in process 2300. Process 2300 is, in 
one embodiment, carried out by a processor under the control of computer- 
readable and computer-executable instructions. The computer-readable and 
computer-executable instructions reside, for example, in data storage features 
such as computer usable volatile memory 506, computer usable non-volatile 
10 memory 508, and/or data storage device 510 of Figure 5. The computer- 
readable and computer-executable instructions are used to control or operate 
in conjunction with, for example, central processing unit 504 of Figure 5 
coupled to or integrated with device 2200 (or 2200a) of Figures 22A and 22B. 

15 For purposes of clarity and brevity, the following discussion and 

examples will specifically deal with video data. The present invention, 
however, is not limited solely to use with video data. Instead, the present 
invention is well suited to use with audio-based data, image-based data, web 
page-based data, graphic data and the like ("media data"). 

20 

In step 2310 of Figure 23, in the present embodiment, device 2200 (or 
2200a) receives video data comprised of a stream of uncompressed video 
frames. In one embodiment, the video data also are comprised of prediction 
error video data generated by a video prediction unit (VPU). As shown by 
25 Figures 8 and 9, respectively, the devices 2200 and 2200a may be coupled to 
a VPU or the VPU may be integral with devices 2200 and 2200a. 

In step 2320 of Figure 23, in the present embodiment, the video data 
are segmented into various regions by segmenter 2202 (Figures 22A and 
30 22B). Segmentation of video data is described above in conjunction with 
Figures 6 (step 604), 10A, 10B, 10C and 10D. As described, the video data 
can be segmented into rectangular regions, non-rectangular regions, and 
overlapping regions, for example. 

35 In step 2330 of Figure 23, in the present embodiment, at least one of the 

regions (or all of the regions) are scalably encoded by encoder 2204 (Figures 
22A and 22B). In one embodiment, each encoded region is encoded into two 
portions: a header portion comprising header data and a payload portion 
comprising scalable video data. The header data provide information about 
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the video data, such as the region within the video frame that the video data 
represent. The header data can also include information that allows a 
transcoder to transcode the video data without decrypting and decoding the 
data, as described previously herein. Scalable encoding is described above 
5 in conjunction with Figure 6 (step 606). 

In step 2340 of Figure 23, in the present embodiment, the scalably 
encoded video data are progressively encrypted by encrypter 2206 (Figures 
22A and 22B). In the embodiment in which data are encoded into a header 
10 portion, the header portion may or may not be encrypted. Progressive 
encryption is described above in conjunction with Figure 6 (step 608). 

In step 2350 of Figure 23, in one embodiment, the scalably encoded 
and progressively encrypted video data are stored in storage unit 2210 
15 (Figure 22B) prior to packetization. 

In step 2360 of Figure 23, in the present embodiment, the scalably 
encoded and progressively encrypted video data are provided to a 
packetizing device 2208 disposed outside of devices 2200 and 2200a 

20 (Figures 22A and 22B). The data can be pushed to packetizing device 2208 
or pulled by packetizing device 2208. In the embodiment in which data are 
not stored, the data are provided to packetizing device 2208 in real time (as 
the data are scalably encoded and progressively encrypted). In the 
embodiment in which the data are stored, the data are provided to packetizing 

25 device 2208 after storage. 

The data provided to packetizing device 2208 may represent the entire 
set of data that was received by devices 2200 and 2200a or a portion thereof. 
That is, in the real time embodiment, at any one of the stages in device 2200, 
30 the data may be reduced because of factors such as the type of channels or 
the type of downstream devices. Similarly, in the storage embodiment, the 
data may be reduced at any one of the stages in device 2200a. Also in the 
storage embodiment, only a portion of the data in storage unit 2210 may be 
provided to packetizing device 2208. 

35 

PACKETIZING DEVICES FOR 
SECURE SCALABLE DATA STREAMING 

With reference to Figures 24, 25A, 25B, 25C and 25D, a process 2400 

for packetizing data in accordance with various embodiments of the present 

40 claimed invention is described. Although specific steps are illustrated in 
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Figure 24, such steps are exemplary, and the present invention is well suited 
to performing various other steps or variations of the steps included in process 
2400. Process 2400 is, in one embodiment, carried out by a processor under 
the control of computer-readable and computer-executable instructions. 
5 Process 2400 is performed by a packetizer device disposed external to 
devices 2200 and 2200a (Figures 22A and 22B, respectively). 

Figures 25A-25D show various embodiments of a packetizing device 
upon which process 2400 of Figure 24 may be implemented. Each of these 

10 embodiments includes a receiver 2502 for receiving a stream of scalably 
encoded and progressively encrypted data from encoding and encrypting 
device 2200 or 2200a of Figures 22A and 22B, respectively. Each of these 
embodiments also includes a packetizer 2506 for packetizing the scalably 
encoded and progressively encrypted data (or a portion thereof) into data 

15 packets. Packetizing device 2208b of Figure 25B includes a memory unit 
2504 for storing scalably encoded and progressively encrypted data prior to 
packetization. Packetizing device 2208c of Figure 25C includes a memory 
unit 2508 for storing secure and scalable data packets subsequent to 
packetization. Packetizing device 2208d of Figure 25D includes a transmitter 

20 2510 for transmitting secure and scalable data packets to a downstream 

device (e.g., a transcoder such as transcoder 1620 of Figure 17). Although the 
embodiments of Figures 25A-25D are separately described in order to more 
clearly illustrate certain aspects of the present invention, it is appreciated that 
combinations of these embodiments may also be used. 

25 

In step 2410 of Figure 24, with reference also to Figures 25A-25D, the 
data are streamed from encoding and encrypting device 2200 or 2200a to 
packetizing device 2208a-d using either a push or a pull approach. The data 
may be received by packetizing device 2208a-d either in real time as they are 
30 encoded and encrypted by device 2200 or 2200a, or after storage on those 
devices. Only a portion of the data may be extracted by or sent to packetizing 
device 2208a-d. For example, packetizing device 2208a-d may extract only 
the amount of data appropriate to the attributes of a downstream channel or 
device. 

35 

In one embodiment, the data received from encoding and encrypting 
device 2200 or 2200a are stored in memory unit 2504 prior to packetization. 
In this embodiment, packetizer 2206 may packetize only a subset of the data 
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stored in memory unit 2504, depending on the attributes of downstream 
channels or devices. 

In step 2420 of Figure 24, again with reference also to Figures 25A- 
5 25D, the data received from devices 2200 and 2200a are formed into data 
packets by packetizer 2206. In the embodiment in which the data include a 
header portion as well as a payload portion, the header portion (scalably 
encoded and either encrypted or unencrypted) is combined and packetized 
with the payload portion (scalably encoded and progressively encrypted). 
10 Packetizer 2208a-d may only packetize a subset of the data, depending on the 
attributes of downstream channels or devices, for example. 

In one embodiment, the data packets received from packetizer 2206 are 
stored in memory unit 2508. In this case, a subset of the data packets may be 
15 retrieved from memory unit 2508, depending on the attributes of downstream 
receiving devices or channels, for example. 

In step 2430 of Figure 24, with reference also to Figures 25A-25D, the 
secure and scaled data packets can be transmitted (streamed) to downstream 
20 receiving devices as described previously herein. In one embodiment, the 
data packets are transmitted to downstream devices using transmitter 2510. 
As described above, only a subset of the data packets may be sent depending 
on downstream attributes and capabilities. 

25 The foregoing descriptions of specific embodiments of the present 

invention have been presented for purposes of illustration and description. 
They are not intended to be exhaustive or to limit the invention to the precise 
forms disclosed, and obviously many modifications and variations are 
possible in light of the above teaching. The embodiments were chosen and 

30 described in order to best explain the principles of the invention and its 

practical application, to thereby enable others skilled in the art to best utilize 
the invention and various embodiments with various modifications as are 
suited to the particular use contemplated. It is intended that the scope of the 
invention be defined by the Claims appended hereto and their equivalents. 
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